Mining of Source Code Concepts and Idioms An Approach based on Clone Detection Techniques

This paper introduces a new view on program source code with a focus on code clone information. An algorithm is presented that transforms source code into an equivalent representation which expresses code redundancies as hierarchical clone classes explicitly. This representation supports program comprehension by pointing out arbitrary programming idioms and the frequencies of their occurrences in the program code. In contrast to most other code clone detection techniques looking for the largest clone fragments only the algorithm shown here provides clone classes for all fragment granularities and include relationships between the clone classes. This makes visible the full range from small idioms that occur frequently in the code to large fragments that are rarely used. Furthermore the provided code representation is specially designed to express parameterized clones as most programming idioms are not an adhesive code block but rather a subtree taken form the middle of an AST, e.g., the concept of a foreach loop without the loop body. The clone information condensed in the representation form shown here can either be used to support idiom-based program understanding. Or it can be used to refactor the most frequent idioms as new language concepts in a DSL in order to compact a given amount of source code.